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Introduction 

High bandwidth applications require large amounts of data transferred to/from storage 
devices at extremely high data rates. Further, these applications often are 'real-time', in 
which access to the storage device must take place on the schedule of the data source, not 
the storage. A good example is a satellite downlink - the volume of data is quite large 
and the date rates quite high (dozens of MB/sec, typically). Further, a telemetry downlink 
must take place while the satellite is overhead; once it passes over the horizon the 
telemetry is lost forever. 


A storage technology which is ideally suited to these types of applications is RAID 
(Redundant Arrays of Independent Disks). The concepts of RAID were presented in an 
academic paper from the University of California’s Berkeley campus in the mid-1980s. 
This paper (often referred to as the 'RAID paper') offered five different architectures, 
colloquially referred to as the 'RAID levels'. Each RAID level, numbered one through 
five, defined a different methodology for using multiple disks grouped together to 
improve performance and offer redundancy. Each of the levels had distinct strengths and 
weaknesses. It is a fallacy to believe the RAID levels with higher numbers (e.g. RAID-4 
versus RAID-2) are superior; the ideal RAID level for an application varies with 
applications - one application may find RAID-1 best suited, RAID-5 for another, and yet 
another application's best choice may be RAID-3. 


RAID Levels 


RAID-1 is classic disk mirroring, in which every disk has a mirror image ot its data 
stored on another disk. This level was the frame of reference in the RAID paper. 
Mirroring has been around for some time, primarily in mainframe computing. Its 
strengths are redundancy and performance. Any single drive in any given data pair may 
fail and the disk system will remain accessible, though at a reduced performance level. 
Because there are two disks for any given piece of data, read performance is quite good as 
any two arbitrary requests for a single logical disk can be serviced simultaneously on two 
physical disks. However, the cost for mirroring is quite high - essentially a 100% 
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premium since every disk is duplicated. The power, cooling, and packaging 
quite high. Reliability is also halved because of the duplication of disks. 


costs are also 


AID-2 and -3 stnpe user data across a group of data drives (typically four or eight drives 
per group) Every block of user data is striped, typically a byte at a time, resulting in all 
the data disks servicing every user data request in parallel. This results in extremely high 
data transfer rates, since multiple disks are transferring data simultaneously. RAID-2 and 

" dlffer m their redundanc y methodologies. RAID-2 uses multiple disks to implement a 
amming error detection and correction code. The codes stored on a RAID-2's 
redundant disks were generated from the data on the data disks. RAID-3 uses a single 
redundant disk to store a error correction code generated by calculating the logical 
exclusive-or of the data on the data disks. Because RAID controller technology doesn't 
require the use of a Hamming code to detect a failed drive, RAID-2 hasn't found 
commercial acceptance as it is more costly than RAID-3. 


RAID-4 and RAID-5 also stripe user data across a group of data drives. However instead 
of striping every block of data across all drives, each block (or sometimes groups of 
blocks) is stored entirely on an individual disk. This results in good transaction 
performance as each disk in the group can service separate requests for individual blocks, 
simultaneously. RAID-4 and -5 differ in the methodology used for storing the error 
correction codes. Both use the exclusive-or code as used in RAID-3. RAID-4 dedicates 
one drive for the error correction codes while RAID-5 rotates the codes throughout all 
drives in the array. RAID-5 has better write performance because of this rotation as there 
is less contention for access to the redundant codes. 


The Right RAID Level for High Bandwidth Applications 


Real-time, high bandwidth applications require the following from disk storage: high 
sustained data transfer rate under all normal operating conditions. Of all the RAID levels 
only RAID-3 fits the profile. 


RAID-4 and RAID-5 don t lit because their performance characteristics are designed for 
delivering a large number of independent requests (high I/Os per second). These RAID 
levels operate best when each disk is servicing a separate request. However, high 
bandwidth applications are characterized by large sequentially stored data sets. For such 
data sets, transfer rate (measured in MB/sec) is the important metric, not I/Os per second. 
Also, both RAID-4 and RAID-5 have severe performance degradations after a drive 
failure, which is considered a normal operating condition in RAID disk arrays. For real- 
time applications this is unacceptable as it is imperative that the RAID subsystem be able 
to service any request, at any time, regardless if there has been a drive failure. 
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RAID-3 fits for two primary reasons. First, because all user data is striped across all 

drives transfer rate is very high. This is true for either reading or writing. In general, a 

RAID-3 disk array will have a sustained transfer rate equal to the product of sustaine 
3er rate of the disks used in the array and the number of data drives in the array^ 
Second, RAID-3 doesn't suffer any performance degradation after a drive fads. - Because 
all of the drives are accessed for each data request, there always is sufficient in 
being transferred from the array that can be combined with the error correction code 
(which is also always transferred on every data request) to generate the failed drive s data. 
Special hardware o'n a RAID-3 controller is able to perform the failed drives data 
reconstruction on-the-fly, with no performance loss. 


Other Factors to Consider 


I„ addition to the medta redundancy inherent in RAID, other subsystem component* 

to operate the entire subsystem in case the other should fai . 

Another subsystem component worth considering 

Fans being a mechanical device, are also prone to failures. A RAID subsystem 
incorporate^ redundant fans to protect against overheating in case of a fan fai ur . 


subsystem down or taking it offline. Most hot swap components will 
canisters or carriers which slide into the RAID subsystem. 


“ “ ; -s xsrsr. sr rr *s:: 

foundTs SCSI-2 It is a 16-bit wide parallel interface which clocks data at 10 MHZfor a 
burs, rate of 20 MB/sec. Sustained rates of over .9 MB/sec am possrble wtth SCSI-2 

RAID-3 disk array. 

The successor to SCSI-2, SCSI-3, includes a performance improvement to 40 MB/sec. 

20 MHa insmad of SCSl-2's 10 MHz. UltraSCSI RAID-3 disk arrays am capab.e of 
sustained data rates in excess of 38 MB/sec. 
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Another intertace which offers excellent high bandwidth performance is Fibre Channel 
I nis is a serial interface which is clorkpH at i kw/ -,1 
capability of 100 MB/ser Fihrf cu- i • , sec w,dl a sustained interface 

. ^ ' ^ re Channel is not physically compatible with SCSI-'’ or ^ 

Channel ^compatible RaS V T t™* Pr ° l ° C ° 1S SUpported - The first Fibre 

p ble RAID-3 disk arrays are becoming available in 1996 with emotin' t 
data rates of nearly 90 MB/sec. h suslamcd 

A good example of high bandwidth RAID-3 disk arrays are those available from Cinrico 
Inc. (Minneapolis. MN). Ciprico offers a full line of high bandwidth disk which 
are well suited to real-time, high bandwidth applications. Ciprico's arrays all offer high 
a transfer rate no performance degradation after drive failures, and media redundancy 
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anety of application s. Table 1 summarizes the capabilities of Ciprico's disk array. 


Model 

Interface 

Burst 

Transfer Rate 

Sustained 
T ransfer Rate 

Redundancy 

Hot Swap 

6500 

UltraSCSI 

40 MB/sec 

38 MB/sec 

Drives 

No ‘ 

6700 

SCSI-2 

20 MB/sec 

19 MB/scc 

Drives 

Power 

YES 

6900 

UltraSCSI 

40 MB/sec 

38 MB/sec 

Drives I 
Power 

YES 

7000 

Fibre 

Channel 

100 MB/sec 

80+ MB/sec 

Drives 

Power 

YES 

Table 1 - Oinri 
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options. Users can select an array which best fits their application. 


Summary 


fonditi^s Wil RAm Pl !m ti0nS high data transfer rates under a11 opiating 

conditions. RAID storage technology, while offering differing methodologies for a 

vane y of applications, supports the performance and redundancy required in § real-time 
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applications. Of the various RAID levels, RAID-3 is the only one which provides high 
data transfer rate under all operating conditions, including after a drive tailure. 
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